Image display system related-information-access tabs

ABSTRACT

A data processing device for providing access to information related to a digital image record includes a display for presenting the digital image record. A user interface system receives an indication of user interaction with the displayed digital image record. A data processing system responsive to the received indication presents tab on the display. The tab displays or emanates near or from an edge of the displayed digital image record or from an edge of a displayed digital image record container including the digital image record. The tab includes an access point configured to allow a user to access a category of information related to content contained within the digital image record, and the tab is labeled according to the category of information for user access.

CROSS-REFERENCE TO RELATED PATENT APPLICATIONS

This application is a Continuation of U.S. application Ser. No.13/191,531, filed Jul. 27, 2011, which is a Continuation of U.S.application Ser. No. 12/195,668, filed Aug. 21, 2008 (now U.S. Pat. No.8,028,246), incorporated herein by reference in its entirety.

FIELD

This invention generally relates to using information from digital imagerecords associated with a user and providing graphical user interfaceutilities for access to related digital content.

BACKGROUND

An increasing number of people are taking advantage of on-line access toand storage of digital records collections, such as digital imagecollections. By using any of a number of various tools that allow accessto images and other records on-line, a user has enhanced opportunitiesfor enjoying, sharing, and using images.

One advantage of digital image storage relates to the amount ofinformation that can be stored in and be associated with the digitalimage data. This information includes increasing amounts of metadatathat are being made available with digital image records obtained fromtoday's digital cameras. For example, digital image records currentlyobtained from a digital camera often have accompanying metadata thatdescribes date, time, and location of images captured, such as from GPSdata obtained by the camera. This information can also includeconditions under which the image was obtained and photographeridentification, and can even have various data about the image contentitself. Standard information data fields or “tags” that can accompany adigital image record are defined, for example, in the Digital StillCamera Image File Format Standard, also termed the EXchangeable ImageFile or EXIF Standard, v.2.1, from the Japan Electronic IndustryDevelopment Association (JEIDA) CP-3451.

Semantic information that can be obtained as metadata for a digitalimage record can include various information obtained from objects inthe image, including data from image analysis tools known in the art,such as various software applications providing scene analysis, objectdetection and recognition, or face detection and recognition. It hasbeen recognized that this capability to extract semantic information canbe useful, for example, in targeted marketing. This capability allows aretailer to present products that may have special appeal to an audiencethat has effectively been “pre-screened” according to images that areowned and viewed by individual members of that audience. That is,targeted marketing aims to identify and reach “pre-screened” consumerswho are likely to have a high level of interest in specific products andservices. Image analysis tools provide another way for marketers ofproducts and services to perform effective pre-screening and to respondto customer interests in a more accurate manner than might otherwise beavailable using other marketing tools.

In a more general sense, the capability to extract content informationfrom within and metadata associated with a digital image record can helpto enhance a user's overall enjoyment of digital images. By obtaininginformation related to images, a computer application can make somereasonable inferences about the user and can use these inferences tooffer other opportunities to the user for functions that access relateddigital content. This can include a range of opportunities for increasedviewing enjoyment, enhanced social interaction, and enhanced marketingfor items or services that are of particular interest.

Conventional methods for targeted marketing and for access to relateddigital content do not provide ways to offer enhanced opportunities thatadapt readily to different image content and allow only a minimum ofuser interaction with image content. Accordingly, there is an ongoingneed for additional or improved techniques for providing user access todigital content that is related to a viewed image.

SUMMARY

According to an aspect of the present disclosure, there is provided adata processing device for providing access to information related to adigital image record, comprising:

a display for presenting the digital image record;

a user interface system for receiving an indication of user interactionwith the displayed digital image record; and

a data processing system responsive to the received indication forpresenting a tab on the display, the tab displaying or emanating near orfrom an edge of the displayed digital image record or from an edge of adisplayed digital image record container including the digital imagerecord, wherein the tab comprises an access point configured to allow auser to access a category of information related to content containedwithin the digital image record, and the tab is labeled according to thecategory of information for user access.

According to another aspect of the present disclosure, there is providedapparatus for providing access to information related to a digital imagerecord on a data processing device comprising:

a data processing device that presents the digital image record on adisplay with at least one tab displaying or emanating near or from anedge of the displayed digital image record or from an edge of adisplayed digital image record container including the digital imagerecord,

wherein the tab comprises an access point configured to allow a user toaccess a category of information related to content contained within thedigital image record.

Some embodiments of the present disclosure include receiving anindication of user-interaction with the displayed digital contentrecord, wherein the receiving step occurs prior to the step ofinstructing presentation of the tab, and wherein the step of instructingpresentation of the tab occurs in response to the receiving step. Theuser-interaction could be a mouse hovering over the displayed digitalimage record, for example.

In addition to the embodiments described above, further embodiments willbecome apparent by reference to the drawings and by study of thefollowing detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be more readily understood from the detaileddescription of exemplary embodiments presented below considered inconjunction with the attached drawings, of which:

FIG. 1 illustrates a system for providing access to information relatedto a digital image record according to an embodiment of the presentinvention;

FIG. 2 is a block diagram of steps for providing access to informationrelated to a digital image record;

FIGS. 3A, 3B, 3C, 3D, and 3E show examples of embodiments in which tabsemanating from a displayed image allow access to information related toimage content; and

FIG. 4 is a schematic block diagram of a networked system for providingaccess to information that is related to image content.

It is to be understood that the attached drawings are for purposes ofillustrating the concepts of the invention and may not be to scale.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention provide access to informationrelated to a digital image record based at least upon an analysis ofimage content from a digital image record that is associated with a userand displayed to the user. Analyzing image content from digital imagerecords provides valuable information about a user. Consequently, suchinformation may advantageously be used for targeted marketing or forother functions that enhance the user's viewing experience.

One type of category of information for user access may be shoppinginformation from a particular product or service provider. The shoppinginformation can be associated with a particular product or serviceprovider, and the shopping information may indicate products or servicesrelated to the content contained within the digital image record andoffered by the particular product or service provider.

The phrase “digital image record,” as used herein, includes digitalstill images as well as digital video. Also, it should be noted that,unless otherwise explicitly noted or required by context, the word “or”is used in this disclosure in a non-exclusive sense. In addition,functions described herein may be implemented as “software” or “softwareprograms”. Those skilled in the art will recognize, however, that theequivalent functions of such software can also be readily executed inhardware.

FIG. 1 illustrates a system 22 for providing access to informationrelated to a digital image record, according to an embodiment of thepresent invention. The system 22 includes a data processing system 26, aperipheral system 30, a user interface system 28, and aprocessor-accessible memory system 24. The processor-accessible memorysystem 24, the peripheral system 30, and the user interface system 28are communicatively connected to the data processing system 26. The dataprocessing system 26 includes one or more data processing devices thatimplement the processes of the various embodiments of the presentinvention, including the example processes of FIG. 2 described herein.

The phrases “data processing device” or “data processor,” as usedherein, include any data processing device, such as a central processingunit (“CPU”), a desktop computer, a laptop computer, a mainframecomputer, a personal digital assistant, a Blackberry™, a digital camera,cellular phone, or any other device for processing data, managing data,or handling data, whether implemented with electrical, magnetic,optical, biological components, or otherwise.

The processor-accessible memory system 24 includes one or moreprocessor-accessible memories configured to store information, includingthe information needed to execute the processes of the variousembodiments of the present invention, including the example processes ofFIG. 2 described herein. The processor-accessible memory system 24 maybe a distributed processor-accessible memory system including multipleprocessor-accessible memories communicatively connected to the dataprocessing system 26 via a plurality of computers and/or devices. On theother hand, the processor-accessible memory system 24 need not be adistributed processor-accessible memory system and, consequently, mayinclude one or more processor-accessible memories located within asingle data processor or device.

The phrase “processor-accessible memory,” as used herein, includes anyprocessor-accessible data storage device, whether volatile ornonvolatile, electronic, magnetic, optical, or otherwise, including butnot limited to, floppy disks, hard disks, Compact Discs, DVDs, flashmemories, ROMs, and RAMs.

The phrase “communicatively connected,” as used herein, includes anytype of connection, whether wired or wireless, between devices, dataprocessors, or programs in which data may be communicated. Further, thephrase “communicatively connected,” as used herein, includes aconnection between devices or programs within a single data processor, aconnection between devices or programs located in different dataprocessors, and a connection between devices not located in dataprocessors at all. In this regard, although the processor-accessiblememory system 24 is shown separately from the data processing system 26,one skilled in the art will appreciate that the processor-accessiblememory system 24 may be stored completely or partially within the dataprocessing system 26. Further in this regard, although the peripheralsystem 30 and the user interface system 28 are shown separately from thedata processing system 26, one skilled in the art will appreciate thatone or both of such systems may be stored completely or partially withinthe data processing system 26.

The peripheral system 30 may include one or more devices configured toprovide digital image records to the data processing system 26. Forexample, the peripheral system 30 may include digital video cameras,cellular phones, regular digital cameras, or other computers. The dataprocessing system 26, upon receipt of digital image records from adevice in the peripheral system 30, may store such digital image recordsin the processor-accessible memory system 24.

The user interface system 28 may include a mouse, a keyboard, anothercomputer, or any device or combination of devices from which data isinput to the data processing system 26. In this regard, although theperipheral system 30 is shown separately from the user interface system28, the peripheral system 30 may be included as part of the userinterface system 28.

The user interface system 28 also includes a display device 10 and aprocessor-accessible memory, or any device or combination of devices towhich data is output by the data processing system 26. In this regard,if the user interface system 28 includes a processor-accessible memory,such memory may be part of the processor-accessible memory system 24even though the user interface system 28 and the processor-accessiblememory system 24 are shown separately in FIG. 1.

Image records stored in a digital image-record collection in theprocessor-accessible memory system 24 may be linked to a variable amountof metadata. This image metadata can include various semantic andstructural information related to the conditions under which the imagewas captured as well as information obtained about image contents. Byway of illustration, metadata for a digital image record can includedate and time of image capture, the capture location (provided by aGlobal Positioning Satellite, GPS, for example), camera owner, cameratype, image resolution, comments from the operator or viewer of theimage, and various data obtained from the image content itself,including information identifying the subject(s) of the image, and otherinformation. Semantic information obtained and stored as metadata for adigital image record can include various information obtained fromobjects in the image, including data from image analysis tools known inthe art, such as various software applications providing objectrecognition or face recognition, as noted previously.

Face detection algorithms are well known and have been described in, forexample, U.S. Pat. Nos. 7,218,759 entitled “Face Detection in DigitalImages” to Ho et al. and commonly assigned U.S. Pat. No. 7,110,575entitled “Method for Locating Faces in Digital Color Images” to Chen etal. both of which are incorporated herein by reference. Face recognitionalgorithms, also known in the art, analyze identified face digital imagerecords to assign an identity to one or more detected faces. Forexample, U.S. Pat. No. 7,142,697 entitled “Pose-Invariant FaceRecognition System and Process” to Huang et al., which is incorporatedherein by reference, describes the use of model digital image records astools for training a neural network to recognize faces in digital imagerecords.

Referring to FIG. 2, there is shown a logic flow diagram of a method forproviding access to information related to a digital image recordaccording to one embodiment of the present invention and using theoverall system arrangement described with respect to FIG. 1. Dataprocessing system 26 has access to a user collection of digital imagerecords in processor-accessible memory system 24. Following the logicflow of FIG. 2, the user accesses an account having an image collection40 that contains digital image records. In a presentation step 100, dataprocessing system 26 provides an instruction to present a digital imagerecord on a display that is part of user interface system 28. In a tabdisplay step 110, data processing system 26 also instructs presentationof at least one tab on the display. The tab, which may have any of anumber of forms, such as those described subsequently, displays oremanates near or from an edge of the displayed digital image record sothat it is visually connected with the displayed image record and actsas an access point that is configured to allow the user to access acategory of information that is related to content that is contained inor otherwise related to the digital image record.

The image content, and any relevant metadata associated with the image,is analyzed to determine image content that can indicate what mightinterest the user who is viewing the image. There are a number oftechniques that can be used to analyze image content, as is familiar tothose skilled in the image analysis arts. For example, variousinformation about viewer interests can be obtained from metadata that isassociated with a digital image record. This can include, for example,time stamp information, Global Positioning Satellite (GPS) informationthat tells the location at which the image was obtained, camera or othersensor type, owner of the image, and other related data. Additionalsemantic information can be obtained by an analysis of objects in theimage. Objects that can be detected include people, animals, locations,various types of equipment, and other image features. These variousitems of information can be collected from the image and used to allowaccess to one or more categories of information that are related.

FIGS. 3A, 3B, 3C, 3D, and 3E show some examples of tabs 20 that emanatefrom edges of a displayed image 14 that appears on display 10. Image 14appears within a rectangular area or similar container on the display.Tabs 20 shown in the examples of FIGS. 3A-3D have the familiarappearance of tabs such as those found on file folders and used invarious graphical user interface (GUI) embodiments. It should be notedthat the term “tab” can be more generally defined as an area or iconhaving a particular function in the interface for accepting user inputand allowing a range of related functions. Tab 20 appears along or nearan outer edge of image 14 so as not to obstruct the image view.

In the embodiment shown in FIG. 3A, tab 20 is labeled “SHOP” and acts asan access point for offering a number of options to the user forentering commands that relate to various product offerings. In theexample shown, a young person proudly displays a fish. Other objects inthe image include a lake or other body of water, a portion of a boat,and various information that suggests a vacation setting. Thisinformation can be used to provide shopping tab 20 with possible itemsof interest based on image content of the displayed image 14 for thepurchase of boating equipment, fishing gear, outdoor equipment, and thelike. Links to vendors who provide this equipment can be displayed asicons 16. These links can display with or without user interaction. Inone embodiment, for example, link icons 16 display when the user mousecursor or other pointer hovers over tab 20. Alternately, link icons 16can be displayed when the cursor hovers over one or more objects withinthe image itself. For example, links to boating equipment may appearwhen the user places the mouse cursor over an image of a boat. Thus,link icons 16 could change dynamically when the same tab 20 displays.Alternately, clicking a mouse button or keyboard key can be used toprovide tab 20 options.

Product offerings are only one of a number of possible options availableusing the tab presentation of the present invention. FIG. 3B shows tab20 providing link icons 16 to any of a number of events that might be ofinterest to the user, again related to digital image record content. Inthis example, “EVENT” tab 20 provides links to related events, such asfishing outings, as well as to related images in the user's digitalimage record collection. Related images could include other images withfish, boating, or outdoors activities, or other images from the samevacation or time frame. Where related images are presented, icon 16 canbe presented in reduced-size or thumbnail form. Clicking on this iconwould then display the selected image as image 14. As a part of thisredisplay process, reconfiguration would be performed, so that imagecontent is again accessed and appropriate tabs 20 displayed.

FIG. 3C shows a “PEOPLE” tab 20 that is directed to family or othersocial associations for the image content. This can include links toother family members, to other images of the same person, or links tofriends or to others who have a similar interest, combining thedetection of a number of objects in image 14, here, both the youngsterand the fishing or outdoors interest.

FIG. 3D shows tab 20 directed to similar locations. “PLACE” tab 20 inthis figure can include links to fishing sites, camps, lakeshoreresorts, tournament sites, and the like, again related to the imagecontent.

FIG. 3E shows an example in which a particular tab 20 becomes activewhen a cursor or other screen pointer hovers over or otherwise indicatethe tab location. FIG. 3E also shows more specific locations for accessto the web site for a particular vendor with additional icons 16.

It can be appreciated that any number of tabs 20 and link icons 16 canbe provided for image content, actuated in any of a number of ways. Tabs20 and link icons 16 can be labeled or include thumbnail images,animations, symbols, corporate or institutional logos, or other textualor pictorial content. Tabs 20 can be labeled with text, as shown in theexamples of FIGS. 3A-3E, or could be labeled using symbols, animations,or other visual objects.

The types of information for any of the categories discussed above inrelation to FIGS. 3A-3E may be organized in hierarchical layers. Forexample, with respect to shopping information discussed in relation toFIG. 3A, a first layer could be the “fishing department”, the next layercould be “categories of products or brands”, and the next layer could bespecific products in those categories. One or multiple layers could bepresented based upon the user's interaction with the tab 20.

FIG. 4 shows an embodiment in which a networked server 42 or otherprocessor communicates with display 10, such as through a localprocessor 44. The user views image 14 from an image collection that isaccessible to networked server 42, whether stored locally or at someother networked location. Server 42 communicates with local processor 44to cause the digital image record to be displayed as image 14 on display10. Then, image analysis utilities, executing on server 42 or on thelocal processor 44, for example, process the image and any associatedmetadata to identify any content that is of interest. FIG. 4 shows thiscontent as a content record 32 that may take any of a number of forms.Information in content record 32 is then provided to server 42 and isused to identify one or more categories of information that may be ofinterest to the user who views image 14. This information can be matchedwith information available to an application on server 42 that maintainsa registry of advertisers who offer products or services to users basedon information gleaned from image content. Server 42 cooperates withprocessor 44 to cause one or more tabs 20 to be displayed, emanatingfrom or near the edge of image 14 and provides the link icons 16 thatallow the user to access other networked information, such as Internetsites or other digital record content.

Embodiments of the present invention allow content information relatedto the digital image record to be used as “feed-through” data, so thatthis information is available to a networked site along with the networkaddress. For example, access to an Internet site may not obtain the mainor “home” page, but may go more directly to vendor information that ispertinent to the identified content. In this way, the user can bedirectly presented an appropriate web page for fishing gear from thesite of a large commercial outdoors outfitter who sells many other typesof outdoor equipment. With this pre-disposed site entry, there would beno need for the user to navigate to an appropriate portion of the website that is of interest, reducing the risk of user confusion orfrustration at not being able to find the specific portion of anetworked site that is of particular interest. The shopping informationthat is accessed thus has products or services more directly related tocontent within the digital image record that is being viewed.

Link icons 16 can also be configured to identify more specific parts ofa web site or multiple parts of a site that might be of interest, as wasshown in FIG. 3E. For the example of FIG. 3E, the user can be presentedwith multiple links to a large site, with various boating, fishing, andoutdoor equipment categories presented for quick and easy user access.

The arrangement and appearance of tabs 20 can be made dynamic, so thatdifferent tabs 20 appear depending upon perceived user focus upon one ormore objects in the displayed image 14. User focus can be ascertained bymouse position or hovering, mouse clicks, touch screen actuation,audible commands, motion sensing, amount of time spent viewing an image,or haptic sensors available at data processing system 26. A standard setof tabs 20 could be provided for each image, or tabs 20 could bedisplayed only where image content appears to relate to differentcategories of interest. For example, for an image containing only asailboat, a tab related to people (such as that shown in the example ofFIG. 3C) may not be appropriate, since there may be no availableinformation related to people in the image. Tabs may alternately appearalong or near any edge of the image within the image area.

The user can be provided with an option to display or to suppress tabs,or to show tabs 20 in a preferred format. Unused tabs 20 may disappearafter a suitable timeout period or appear only at intervals, such asonly after an image is viewed for 10 seconds or longer, for example. Ortabs 20 may be displayed only upon receipt of a suitable prompt from theuser, such as a mouse click or hovering or touch screen actuation.

Where icons 16 indicate other image content, clicking on an icon, suchas a thumbnail image, reconfigures or refreshes display 10 in someembodiments, so that the newly selected image now displays as image 14.The necessary image analysis and display of tabs 20 appropriate for thatimage content is also carried out along with the screen refreshoperation.

User digital image record collection 40 (FIG. 2) may include a pluralityof digital image records, which may be still or video images. Suchdigital image records may have been captured using various types ofdigital cameras or may have been scanned from conventional photographsand stored digitally. One of ordinary skill in the art will appreciatethat the invention is not limited to the manner in which the digitalimage records were acquired. Digital image records in digital imagerecord collection 40 may have accompanying metadata ranging fromdate/time stamp information to full-fledged data on imaging conditions,GPS location, type of capture device employed, image capture conditions,and the like.

It is to be understood that the exemplary embodiments are merelyillustrative of the present invention and that many variations of theabove-described embodiments can be devised by one skilled in the artwithout departing from the scope of the invention. It is thereforeintended that all such variations be included within the scope of thefollowing claims and their equivalents.

The invention has been described in detail with particular reference tocertain preferred embodiments thereof, but it will be understood thatvariations and modifications can be effected within the spirit and scopeof the invention.

PARTS LIST

-   10 display-   14 image-   16 icons-   20 tab-   22 system-   24 processor-accessible memory system-   26 data processing system-   28 user interface system-   30 peripheral system-   32 content record-   40 image record collection-   42 networked server-   44 local processor-   100 instruct presentation of image record step-   110 instruct display of tab as access point step

What is claimed is:
 1. A device comprising: a user interface systemconfigured to receive an indication of user interaction with a displayeddigital image; and a processing system configured to present a menuassociated with the displayed digital image in response to the receivedindication, wherein the menu emanates from the displayed digital image,wherein the menu comprises an access point configured to allow a user toaccess a category of information related to content contained within thedigital image, wherein the category of information includes informationabout a similar location or an object related to a location of a scenedepicted in the displayed digital image, and wherein the menu is labeledaccording to the category of information.
 2. The device of claim 1,wherein the device is a handheld device and the user interaction is atouch screen actuation or an audible command.
 3. The device of claim 2,wherein the handheld device includes a personal digital assistant, asmartphone, a digital camera, or a cellular phone.
 4. The device ofclaim 2, wherein the handheld device includes a cellular device.
 5. Thedevice of claim 1, wherein the menu emanates from an edge of thedisplayed digital image.
 6. The device of claim 1, wherein the menu doesnot obstruct an image view of the displayed digital image.
 7. The deviceof claim 1, wherein the processing system is configured to present aplurality of menus emanating from the displayed digital image, whereineach of the plurality of menus corresponds to a different category ofinformation.
 8. The device of claim 7, wherein at least one of theplurality of menus corresponds to a category of information thatincludes shopping information relating to the content contained withinthe displayed digital image.
 9. The device of claim 7, wherein at leastone of the plurality of menus corresponds to a category of informationthat includes event information relating to the content contained withinthe displayed digital image.
 10. The device of claim 7, wherein at leastone of the plurality of menus corresponds to a category of informationthat includes personal information relating to the content containedwithin the displayed digital image.
 11. The device of claim 7, whereinat least one of the plurality of menus corresponds to a category ofinformation that includes information about other images from a samevacation or time frame as the displayed digital image.
 12. The device ofclaim 1, wherein the user interaction includes a mouse hover.
 13. Thedevice of claim 1, wherein the user-interaction includes a mouse click,a physical touch of a touch-screen display, a point using a motionsensor, a remote control device, a voice command, a mouse-button click,or a keyboard key press.
 14. The device of claim 1, wherein the similarlocations comprise at least one of fishing sites, camps, lakeshoreresorts, or tournament sites.
 15. A method comprising: receiving, at aprocessing system, an indication of user interaction with a displayeddigital image; and in response to the received indication, presenting,by the processing system, a menu associated with the displayed digitalimage, wherein the menu emanates from the displayed digital image,wherein the menu comprises an access point configured to allow a user toaccess a category of information related to content contained within thedigital image, wherein the category of information includes informationabout a similar location or an object related to a location of a scenedepicted in the displayed digital image, and wherein the menu is labeledaccording to the category of information.
 16. The method of claim 15,wherein the menu emanates from an edge of the displayed digital image.17. The method of claim 15, wherein the menu does not obstruct an imageview of the displayed digital image.
 18. The method of claim 15, furthercomprising presenting a plurality of menus emanating from the displayeddigital image, wherein each of the plurality of menus corresponds to adifferent category of information.
 19. A non-transitory computerreadable medium having instructions stored thereon, the instructionscomprising: instructions for receiving an indication of user interactionwith a displayed digital image; and instructions for presenting, inresponse to the received indication, a menu associated with thedisplayed digital image, wherein the menu emanates from the displayeddigital image, wherein the menu comprises an access point configured toallow a user to access a category of information related to contentcontained within the digital image, wherein the category of informationincludes information about a similar location or an object related to alocation of a scene depicted in the displayed digital image, and whereinthe menu is labeled according to the category of information.
 20. Thenon-transitory computer-readable medium of claim 19, wherein the menuemanates from an edge of the displayed digital image such that the menudoes not obstruct an image view of the displayed digital image.